The present disclosure relates generally to imaging and visualization, and, more particularly, to a generalized application programming interface (API) for rendering graphics on a variety of output devices. Example output device types include, but are not limited to: two-dimensional displays; three-dimensional displays such as volumetric, multi-view, and holographic displays; and two- and three-dimensional printers.
Modern graphics environments must solve the problem that the application software generally runs on separate hardware from the rendering algorithms. Since off-the-shelf personal computers (PCs) are not yet specialized for spatial three-dimensional (3-D) rendering, the process separation is generally more complicated than sending the data across the peripheral component interface (PCI)-express bus. The Chromium architecture is a prior attempt to solve this problem. Chromium abstracts a graphical execution environment. However, the binding between an application, rendering resources and display is statically determined by a configuration file. Therefore, applications cannot address specific rendering resources.
To ensure applicability to a broad range of users, the software environment for the display should cater to at least three classes of software applications. Consider, for example, the case in which the output device is a volumetric 3-D display such as the Perspecta® Spatial 3-D Display (Actuality Systems, Inc., Bedford, Mass.) described in U.S. Pat. No. 6,554,430. The first class of software applications includes legacy applications that were written without foreseeing integration with a volumetric display. It is helpful to be able to use legacy applications because a large number of them exist which perform useful and challenging visualization work. A second class of software applications includes ported applications that have had some minor adjustment by an application vendor to accommodate a volumetric display. Since volumetric displays are not already in widespread use, application vendors may take a risk when they modify their software for these displays. To get the broadest range of applications, it is important to minimize the risk to application vendors by providing easy-to-use extensions to existing graphics APIs. A third class of software applications includes native applications that are written to take full advantage of the capabilities of volumetric displays.
Continuing this example, the software environment for a volumetric display must provide the ability to efficiently handle large datasets and be flexible enough to apply a broad range of rendering effects. Previous low-level graphics APIs were not designed with 3-D displays in mind. They do not provide the abstractions needed to efficiently target the variety of two-dimensional (2-D) and 3-D displays that are now becoming available. The benefit to the visionary application vendors that adapt to new display technologies should be maximized by providing a consistent abstraction to a variety of emerging display types.
A well-known graphics API is the Open Graphics Language (OpenGL) API. In some cases it is possible to adapt an OpenGL driver (e.g., the OpenGL Driver from Actuality Systems, Incorporated) with little effort, to yield interoperability with a legacy software application. Many off-the-shelf applications are fully functional and no additional work is required. However, sometimes it is not possible to adapt all of the functions of a complex application or one that uses specialized rendering techniques. Customers are now looking for a higher level of integration with these sophisticated applications. OpenGL does not sufficiently abstract the process of rendering to suit volumetric displays to support this higher level of integration.
One problem with the OpenGL API is that it includes assumptions that are only appropriate for a flat, 2-D monitor. For example, the depth of the model within the rendering window is not defined. A second problem with OpenGL is that it does not abstract a large enough range of rendering operations. Volume rendering is an important function in many professional applications, and it may grow in importance in recreational applications. The parameters to control a volume rendering operation are similar for all display types, and there are flexible techniques for implementing the operations on existing graphics hardware. It would be desirable for a display-agnostic interface to abstract volume rendering.
Furthermore, volumetric displays cannot be completely served by existing graphical user interface (GUI) infrastructures. Actuality Systems, Incorporated has defined a volume manager to organize the new functions required by volumetric applications. One of the first features of this volume manager is to provide an application-independent 3-D pointer. See, for example, U.S. patent application Ser. No. 10/941,452 to Napoli et al., of common assignment herewith and U.S. Patent Application No. 2004/0135974 A1 to Favalora et al., also of common assignment herewith.
In summary, current software architectures for 3-D display systems suffer from at least the following limitations. Applications cannot address remote or distributed resources. Such resources are necessary for displays where ready-made rendering hardware is not available for PCs. Also, applications can only address assets that are stored in a few prescribed formats. Imaging application software cannot access datasets with specialized encodings unless the application software includes a reformatting step. Reformatting reduces the ultimate performance of the application. A further limitation is that legacy, ported, and native application classes are not handled in a uniform way. Existing 3-D graphics APIs are designed with assumptions that are only valid for flat, 2-D monitors. For example, the appearance of a scene is only defined from a single given viewpoint. Another limitation is that existing 3-D graphics APIs do not completely abstract rendering algorithms. For example, a volume rendering operation cannot be specified. A further limitation is that existing software architectures do not provide support for 3-D user interfaces.